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Abstract 

As automation and advanced technologies are introduced into transport systems ranging from the Next 
Generation Air Transportation System termed NextGen, to the advanced surface transportation systems as 
exemplified by the Intelligent Transportations Systems, to future systems designed for space exploration, 
there is an increased need to validly predict how the future systems will be vulnerable to error given the 
demands imposed by the assistive technologies. One formalized approach to study the impact of assistive 
technologies on the human operator in a safe and non-obtrasive manner is through the use of human 
performance models (HPMs). HPMs play an integral role when complex human-system designs are 
proposed, developed, and tested. One HPM tool termed the Man-machine Integration Design and 
Analysis System (MIDAS) is a NASA Ames Research Center HPM software tool that has been applied to 
predict human-system performance in various domains since 1986. MIDAS is a dynamic, integrated HPM 
and simulation environment that facilitates the design, visualization, and computational evaluation of 
complex man-machine system concepts in simulated operational environments. The paper will discuss a 
range of aviation specific applications including an approach used to model human error for NASA’s 
Aviation Safety Program, and “what-if’ analyses to evaluate flight deck technologies for NextGen 
operations. This chapter will culminate by raising two challenges for the field of predictive HPMs for 
complex human-system designs that evaluate assistive technologies: that of (1) model transparency and 
(2) model validation. 

1 Introduction 

Human Performance Models (HPMs) have traditionally been used to predict sensory processes (Gawron, 
Laughery, Jorgensen, & Polito, 1983), aspects of human cognition (Newell, 1990), and human motor 
responses to system tasks (Fitts & Posner, 1967; Gore, 2001; Hudlicka & Corker, 2001). HPM tools are 
currently undergoing a developmental shift, now being more sensitive to situations that confront a virtual 
human in systems similar to human-in-the-loop (HITL) situations. HPMs and the human performance 
modeling process have attempted to integrate operator characteristics (cognitive, attentional, and 
physical) with environmental characteristics to more accurately represent human-system operations with 
new, augmented technologies. The growth in HPMs has been to examine human performance in systems 
including system monitoring (thereby taking information in from the environment) as opposed to the 
closed-loop view of the human as a mathematical relationship between input and output to a system. 
These hybrid models that combine closed-loop performance (continuous control), open-loop performance 
(discrete control) and critical decision-making have been undertaken to represent the “internal models and 
cognitive function” of the human operator in complex control systems. These hybrid systems involve a 
critical coupling among humans and machines in a shifting and context sensitive function. 


1.1 Using Human Performance Models (HPMs) for Technology Development 

Modeling can play a role in all phases of new technology development from concept development, 
through the refinement, and deployment process. HPMs provide a flexible and economical way to 
manipulate aspects of the operator, automation, and task environment to represent the manner that a 
human engages with the technology under development (Gore, 2002, 2008; Gore, Hooey, Foyle, & Scott- 
Nash, 2008; Hooey & Foyle, 2008). HPMs have arisen as viable research options due to decreases in 
computer costs, increases in representative results, and increases in model validity. They are especially 
valuable because the computational predictions can be generated early in the design phase of a product, 
system or technology to formulate procedures, training requirements, and to identify system 
vulnerabilities and where potential human-system errors are likely to arise. The model development 
process allows the designer to formally examine many aspects of human-system performance with new 
technologies to explore potential risks brought to system performance by the human operator (Gore & 
Smith, 2006). Often this can be accomplished before the notional technology exists for human-in-the- 
loop (HITL) testing (Gore, 2000). More comprehensive results and conclusions can be drawn about 
technologies being introduced into complex operational environments when used in a cooperative and 
iterative fashion with HITL simulations. Furthermore, using HPMs in this manner is advantageous 
because risks to the human operator and costs associated with system experimentation are greatly 
reduced: no experimenters, no subjects and no testing time (Elkind, Card, Hochberg, & Huey, 1989; 
Gore, 2001). HPMs can be used to conduct system robustness testing to evaluate the system from the 
standpoint of potential deviations from nominal procedures to determine the impact on the performance of 
the human and the system (“what-if ’ testing) (Gore & Smith, 2006; Hooey & Foyle, 2008). 

2 The Man-machine Integration Design and Analysis System 

MIDAS is a dynamic, integrated human performance modeling and simulation environment that 
facilitates the design, visualization, and computational evaluation of complex man-machine system 
concepts in simulated operational environments (Gore, 2008). MIDAS symbolically represents many 
mechanisms that underlie and cause human behavior (Gore & Jarvis, 2005). MIDAS combines graphical 
equipment prototyping, dynamic simulation, and HPMs to reduce design cycle time, support quantitative 
predictions of human-system effectiveness, and improve the design of crew stations and their associated 
operating procedures. 

2.1 History 

MIDAS has undergone two paths in its development. The first path termed Air MIDAS focused on 
specific behaviors in complex human-system interaction and has been applied specifically to aviation 
operations (Gore, 2002). This development path was entirely code-based with no visualization capability. 
The second path, termed the NASA MIDAS and currently MIDAS v.5, focused on cross-domain 
capability (Gore & Smith, 2006), cognitive behavior model augmentations (Gore, et al., 2008; Gore, 
Hooey, Wickens, & Scott-Nash, 2009) and has been applied to a variety of transportation domains Human 
performance models have produced validated predictions of human performance within complex 
operating environments ranging from rotorcraft (Atencio, 1994, 1998), nuclear power plan operations 
(Corker, 1994), space operations (Gore & Smith, 2006), and commercial aviation (Gore, Hooey, Wickens, 
& Scott-Nash, 2009; Hooey, et al., 2010). The NASA MIDAS path contains a comprehensive 
visualization capability associated with the physical and cognitive operations in their respective contexts. 
MIDAS links a virtual human, comprised of a physical anthropometric character, to a computational 
cognitive structure that represents human capabilities and limitations. The cognitive component is 
comprised of a perceptual mechanism (visual and auditory), memory (short-term, long-term working, and 
long-term), a decision maker and a response selection architectural component. The complex interplay 
among bottom-up and top-down processes enables the emergence of unforeseen, and non-programmed 
behaviors. MIDAS can suggest the nature of pilot errors, and highlight precursor conditions to error such 
as high levels of memory demand, mounting time pressure and workload, attentional tunneling or 



distraction, and deteriorating situation awareness (SA). MIDAS provides a flexible and economical way 
to manipulate aspects of the operator, automation, and task environment for simulation analyses. 

2.2 MIDAS v5 Architecture 

Figure 1 illustrates the model’s organization and flow of information among the model’s components. 
MIDAS inputs (Figure 1, left column) include the operational environment (e.g., flight profdes, scenario 
objects and events, etc), the operator tasks and operator process models (e.g., algorithms that represent 
operator characteristics such as expertise). The MIDAS processing model (Figure 1, middle column) is 
comprised of a task manager model that schedules tasks to be completed, definitions of the state of 
models within the physical simulation, a library of “basic” human primitive models that represent 
behaviors required for all activities such as reach, and cognitive models such as operator perception. The 
MIDAS output model (Figure 1, right column) generates a runtime display of the task network, the 
anthropometry as well as mission performance. 



Figure 1. MIDAS Structural Composition and Flow: The agents that make up the MIDAS behavior set are represented as well as 
the linkage of information passed from one agent to another (adapted from Gore and Smith, 2006). 

2.2.1 MIDAS Input 

Tasks are triggered by information that flows from the environment, through a perception model, to a task 
network representation of the procedures that then feeds back into the environment. Tasks are 
characterized by several defining parameters that include conditions under which the task can be 
performed (e.g., beginning, ending, and wait-for clauses), its relative priority with respect to other tasks, 
an estimate of its duration for scheduling, its interruption specifications, and the resource required to 
perform the task defined according to the Modified TAWL (McCracken & Aldrich, 1984; Mitchell, 
2000). 

2.2.2 MIDAS Processing 

2.2.2. 1 MIDAS Perception 

MIDAS represents perception as a series of stages that information must pass through in order to be 
processed. The perception model includes visual and auditory information. Visual perception in MIDAS 
depends on the amount of time the observer dwells on an object and the perceptibility of the observed 




object. The perception model computes the perceptibility of each object that falls into the operator’s field 
of view based on properties of the observed object, the visual angle of the object and environmental 
factors. In the current implementation of MIDAS, perception is a three-stage, time-based perception 
model (undetected, detected, comprehended) for objects inside the workstation (e.g., an aircraft cockpit) 
and a four-stage, time-based perception model (undetected, detected, recognized, identified) for objects 
outside the workstation (e.g., taxiway signs on an airport surface). The model computes the upper level 
of detection (i.e., undetectable, detectable, recognizable, identifiable for external objects) that can be 
achieved by the average unaided eye if the observer dwells on it for a requisite amount of time. For 
example, in a low-visibility environment, the presence of an aircraft on the airport surface may be 
‘detectable’ but the aircraft company logo on the tail might not be ‘recognizable’ or ‘identifiable’ even if 
he/she dwells on it for a long time. 

22 . 2.2 MIDAS Memory 

Tasks from the MIDAS input process also require knowledge held either in the operator’s memory 
(working, long-term working, and long-term) or available from the environment to be consulted and used 
to determine subsequent tasks to be completed (Gore, Hooey, Wickens, & Scott-Nash, 2009). Memory is 
represented as a three stage, time decay model 1 . The stages are working memory (WM), long-term 
working memory (LT-WM), and long-term memory (LTM). The decay rates cause memory to be above 
or below a “retrievability” threshold based on the time since the information was last accessed. The 
retrievability thresholds incorporated into MIDAS are 5 seconds for WM and 5 minutes (300 seconds) for 
LT-WM. The WM decay rate is faster than the LT-WM decay rate. Information that falls below the 
retrevability threshold is forgotten. This causes the perception level to be set to Undetected for external 
visual and auditory information or Unread for internal visual information. Newly perceived and recently 
refreshed attributes will be retained in LT-WM only if a node with newly updated or referenced attributes 
leaves WM before its attributes have decayed below the retrievability threshold. An operator may retain 
newly perceived information after it leaves WM, at least for a while, until it decays below the LT-WM 
retrievability threshold. If the information necessary for activity performance is available, and its priority 
is sufficient to warrant performance, then the schedule within the model operates according to heuristics 
that can be selected by the analyst. In most cases the heuristic is to perform activities concurrently when 
that is possible, based on knowledge and resource constraints. 

2.2.2.3 MIDAS Visual Attention 

MIDAS’ attention-guiding model operates according to the SEEV model (Gore, Hooey, Wickens, & 
Scott-Nash, 2009). SEEV estimates the probability of attending, P(A), to an area of interest in visual 
space, as a linear weighted combination of the four components - salience, effort, expectancy, and value. 
Attention in dynamic environments is driven by the bottom-up capture of Salient (S) events (e.g., a 
flashing warning on the instrument panel) and inhibited by the Effort (E) required to move attention (e.g., 
a pilot will be less likely to scan an instrument located at an overhead panel, head down, or to the side 
where head rotation is required, than to an instrument located directly ahead on a head-up display (HUD) 
(Wickens, Goh, Helleberg, Horrey, & Talleur, 2003). The SEEV model also predicts that attention is 
driven by the Expectancy (EX) of seeing a Valuable (V) event at certain locations in the environment. 
SEEV is driven per the following equation 2 : 

P(A) = s*S -ef*EF + ex*EX + v*V. (1) 


1 In contrast to MIDAS v5, memory in Air MIDAS is represented as a two-stage model of WM and LTM (Baddeley 
& Hitch, 1974). 

2 Properties of a display or environment are illustrated by coefficients in uppercase characters, while those in lower 
case describe the weight assigned to those properties in the control of an operator’s attention (Wickens, et al., 2008). 



The four SEEV parameters drive the visual attention around an environment such as the dynamic cockpit 
in a computational version of this model. For example, the simulated eyes following the model will fixate 
more frequently on areas with a high bandwidth (and hence a high expectancy for change), as well as 
areas that support high-value tasks, like maintaining stable flight (Wickens, Hooey, Gore, Sebok, & 
Koenecke, 2010). SEEY has been under development since 2001 and has been extensively validated with 
empirical human-in-the-loop data from different domains (Gore, Hooey, Wickens, Sebok, et al., 2009; 
Wickens & McCarley, 2008). 

The integration of the SEEV model into MIDAS v5 allows dynamic scanning behaviors by calculating 
the probability that the operator’s eye will move to a particular AOI given the tasks the operator is 
engaged in within the multitask context. It also better addresses allocation of attention in dynamic 
environments such as flight and driving tasks. 


2.2.3 MIDAS Output 

The MIDAS outputs include the task network model, anthropometric model and Computer Aided Design 
(CAD) visualizations (using the jack™ software), timelines of workload and SA, and risk vulnerabilities 
as inferred from timeline violation of optimal response times, workload spikes, or SA violations. The 
complex interplay among bottom-up and top-down processes enables the emergence of unforeseen, and 
non-programmed behaviors (Gore & Smith, 2006). MIDAS can suggest the nature of operator errors, and 
highlight precursor conditions to error such as high levels of memory demand, mounting time pressure 
and workload, attentional tunneling or distraction, and deteriorating SA. An illustration of the integration 
of the different models in a recent aeronautics simulation completed with MIDAS v5 can be found in 
Figure 2. 
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Figure 2. MIDAS’ Environment, Task, and Anthropometric Models. 


jack' is maintained by Siemens PLM Solutions. 



3 Human Performance Modeling of Next Generation Air Transportation 
Systems 

The current air traffic control (ATC) system in the United States will not be able to manage the predicted 
two to three times growth in air traffic (JPDO, 2009). The Next Generation Air Transportation System 
(NextGen) is a future aviation concept that has as its goals to increase the capacity, safety, efficiency, and 
security of air transportation operations (JPDO, 2009). Two MIDAS HPMs will be highlighted. The first 
illustrates a human error model of an aviation surface-related application that uses some candidate 
NextGen concepts as generated by Air MIDAS, and the second illustrates a recent application of NASA’s 
MIDAS in the context of NextGen approach and land operations. 

3.1 Human Error Modeling 

New conceptual designs especially those being developed for complex systems are likely to incorporate 
technologies that utilize or rely on a human’s cognitive capabilities. New conceptual designs often 
incorporate automation to assist the human operator in their task performance. Automation increases 
precision and economy of operations but can have the unanticipated effect of increasing a human 
operator’s cognitive, perceptual and attentional workload (Sarter, Woods, & Billings, 1997). The increase 
in workload often negates some of the benefits afforded to the system from the use of automation. 
Operators may miss critical events in the environment due to a number of unexpected human-automated 
systems issues such as unevenly distributed workload, new attentional demands, and new coordination 
demands among operators (Sarter, et al., 1997). When critical physical events are missed, no response is 
possible and human errors occur (Hollnagel, 1993, 2000). 

Quantifying human error in a manual control representation is a task that can be modeled quite simply by 
control input-output mechanisms exemplified by the servo-mechanisms developed in the mid-1940's 
(Craik, 1947). The identification of the underlying reasons for the error exhibited by human performance, 
however is much more difficult. The reasons for inaccurate human performance could be the result of 
inadequate design of the physical work environment or could be due to the human cognitive component. 
Many theories exist on human error causality and all have some merit as they attempt to quantify the 
measurement of human error (Hollnagel, 1993; Hollnagel & Woods, 2005; Norman, 1981; Rasmussen, 
1976; Reason, 1990). 

3.1.1 Modeling Human Error 

An integrated flight deck model of ramp navigation and gate-approach at the Chicago O’Hare Airport 
(ORD) was used to produce human error performance when technological introductions that took the 
form of augmented flight deck concept displays (T-NASA; Foyle, Andre, & Hooey, 2005) were made to 
current day operations using the Air MIDAS HPM (Gore, Verma, Jadhav, Delnegro, & Corker, 2002 ). In 
order to generate a sufficiently valid model of error predictions, the equipment (physical aircraft), the 
crewstation and the external environment were modeled at varying levels of fidelity depending on the 
importance of the information for updating the operator’s world. The control modes in Air MIDAS that 
had the potential of being sensitive to manipulations include memory errors and their effect on the 
simulated crew’s internal representation, termed updateable world representation (UWR). Three types of 
error mechanisms were represented. 

- The first error type, declarative memory errors, included errors that occurred when virtual operators 
forgot the active procedure as a result of having too many procedures of the same type operating at 
the same time. The occurrence of this error was modeled by scheduling the simulation to cause 
multiple competing behaviors to occur concurrently and invoke the procedure scheduler (dropped 
tasks = memory loss). 

- The second error type, memory load errors, occurred as a result of information competing for WM 
space. When there were a number of items needing to occupy WM, one item in WM needed to be 



shifted out of the limited capacity store by the subsequent information from the pilot or from the 
controller communication. This information was lost if it not written down to a location from an 
actively available list from which the operator was able to visually encode the information (for 
example, a taxi clearance). 

- The third type of error, UWR discrepancy errors, occurred when there was a worldview inconsistency 
between two virtual operators. This error occurred when one virtual operator erroneously “thought” a 
different virtual operator had received information. This UWR discrepancy was a possible cause of 
and a response to the occurrence of an error. 

The errors represented above were created by increasing the number of items in WM and by increasing 
information transfers through active procedures. Each type of error emerged as a result of the scenario 
requirements and demands placed on the virtual operators and each type of error was not created 
deterministically. 

The Air MIDAS software operated with different human operator model memory decay rates of 120 
seconds and 40 seconds. Environment triggers (e.g., turns, signs, ATC calls) elicited the human 
performance. Error rates as measured by missing turns, operator performance times and workload were 
output from the HPM. Each of these individual elements was considered to be critical in the analysis of 
error for determining the conditions under which the error rate changes in response to operational 
changes. In the Air MIDAS model, errors occurred as a result of increasing decay rates and reducing 
memory capacity. As this effort predicts, the model loading factors appear to have an impact on the 
performance of the forgetting mechanism and on invoking the rule guiding the response to the forgetting 
function. This demonstrated that the computational mechanisms within the Air MIDAS software are 
attempting to replicate the operations of humans when humans are faced with a forgotten piece of 
information. When there are a number of items occupying WM, one item in WM is shifted out of the 
limited capacity store by the subsequent information from the pilot or from the controller communication. 
Each type of error was therefore not created deterministically; rather each error type emerged based on 
the environmental requirements and loads that were associated with the performance requirements of the 
agent. A prediction for increased auditory and cognitive demands as time in the scenario increased (as the 
virtual operator approached the second turn) was also found. The Air MIDAS model provided useful 
information for identifying risk factors that increase the probability of error and was useful for providing 
some insight into mitigation strategies when errors occur. Air MIDAS did not however predict error 
rates. 

3.1.2 “What-If“ Next Gen Approach and Landing Application 

MIDAS v5 has been applied to examine a NextGen approach to land concept termed the very closely 
spaced parallel approach/operations (VCSPA/O). Based on Raytheon’s Terminal Area Capacity 
Enhancement (TACEC) parallel approach procedures, VCSPA/O requires that runway spacing be reduced 
(Verma, Lozito, & Trott, 2008). This reduction in distance between the runways increases the likelihood 
of wake vortex incursion during independent simultaneous operations. VCSPA/O requires that a safe and 
proper breakout maneuver be calculated and presented via new displays to the cockpit crew (Verma, et 
ah, 2008). In order to evaluate the VCSPA/O concept, two MIDAS v5 models were generated. The first 
was a Simultaneous Offset Instrument Approach (SOIA) model that contained the current-day 
procedures, and the second was a NextGen VCSPA/O model that contained advance displays of traffic 
and wake information in the cockpit and a modification to the roles and responsibilities of the flight crew 
and ATC modeled operators. The advanced technology in the “NextGen” VCSPA/O condition enabled 
closer separation in low visibility, lower landing minima, autoland capability, and enhanced wake and 
traffic data. This MIDAS model involved over 500 tasks and culminated in a verifiable model of 
approach and land operations (vetted during the model building process by Subject Matter Experts; 
SMEs). One hundred Monte Carlo model runs were generated by MIDAS. Performance profiles along the 
variables of operator workload, visual attention, and cockpit alert detection times for both the captain and 



first officer during the descent, approach, and land phases of flight were collected. Both conditions were 
run in IMC with no out the window (OTW) visibility. The SOIA flight crew broke out of the clouds at 
2100’ and maintained separation from traffic and monitored runway alignment OTW. The VC SPA 
modeled flight crew monitored traffic separation and wake information on the ND throughout the 
approach and broke out of the clouds at 100’. This model effort illustrated the “what-if’ simulation 
capability within MIDAS. The “what-if’ approach was completed when MIDAS was exercised with one 
set of displays and procedure sets designed to represent current day operations and roles followed by a 
second simulation with an alternate set of displays and procedures encoded to represent the NextGen 
displays and expected procedures. 

Important insights regarding the impact of NextGen VCSPA/O operations on pilot workload, visual 
attention, and alert detection times were revealed through this research. The MIDAS model predicted 
increased workload during descent and initial approach due to increased information available (traffic, 
weather, wake) on the flight deck but reduced workload during final approach and land due to automated 
landing procedures and ease of information retrieval (traffic and runway alignment) in the NextGen 
condition. Technologies that shift the workload demands away from the visual modality using auditory 
and haptic displays should be pursued as NextGen operations may tax the visual and cognitive-spatial 
channels to a greater extent than current day operations during specific phases of flight. Furthermore, the 
NextGen condition suggested a more balanced workload across the descent, approach, and land phases of 
High than current day operations. In terms of visual attention, NextGen condition may draw visual 
attention to the navigation display (ND), which suggest that the pilots will more likely be heads-down 
during the critical minutes before touchdown (TD). Increased head-down time within the cockpit due to 
the presence of additional, more salient, information, may draw pilots’ attention into the cockpit at 
inopportune times leaving pilots vulnerable to external hazards (other aircraft or obstacles on the runway, 
terrain). It is important to remember that other instantiations of the VCSPA/O concept (with different 
operational requirements) may reveal different human-system vulnerabilities. 

This MIDAS v5 effort led to a greater awareness of potential parameters such as the change in roles and 
responsibilities in the NextGen that should be included in system designs and enabled the research 
program to visualize the interactions that are likely in future NextGen operations. It is anticipated that 
additional validation approaches will be developed and applied to the VCSPA/O require an increased 
number of alternative closely spaced operations for additional “what-if’ scenarios including alternative 
pilot roles and responsibilities, and information requirements. 

4 Conclusion 

Automation often changes the nature of the human’s role in the system. Therefore, as automation and 
technologies are developed, it becomes increasingly important to predict how the human operator 
perceives and responds to the automation. MIDAS has proven useful for identifying general human- 
system vulnerabilities and cross-domain error classes and for recommending mitigation strategies and 
system re-designs to account for the vulnerable areas, or risks, in system design (Gore & Smith, 2006). 
Fundamental design issues can therefore be identified early in the design lifecycle, often before hardware 
simulators and HITL experiments can be conducted. HPMs are most useful when used cooperatively with 
HITL simulations to supplement the HITL research. HPMs like MIDAS provide an easy to use and cost 
effective means to conduct experiments that explore "what-if 1 questions about concepts of operation 
(CONOPS) in specific domains of interest. 

A number of significant challenges exist for the state of the art in HPMs, two of which will now be 
highlighted. 

4.1 Transparency 

The first challenge relates to model transparency. Model transparency refers to the ability to comprehend 
the relationships that exist among the models being used in the simulation, the performance of the models 



in the simulation, which models are triggering in the model architecture, and whether the model is 
behaving as the model developer would expect (Gore, 2008). Other researchers refer to this as model 
traceability, model behavior visibility, model verifiability, and model interpretability (Elkind, et ah, 1989; 
Gluck & Pew, 2005; Hooey & Foyle, 2008; Napierski, Young, & Harper, 2004). Transparency in 
integrated HPMs is needed to support model verification, validation, and credibility. However, model 
transparency can be difficult to attain because of the complex interactions that can exist among the 
cognitive, physical, environment and crew station models, and because the cognitive models embedded 
within integrated HPMs produce behaviors that are not directly observable. Three types of transparency 
that the MIDAS researchers have found useful to understand, interpret, and increase the confidence in the 
complex models’ output include transparency of the input, transparency of the integrated architecture, and 
transparency of the output (Gore, et al., 2008). 

4.2 Validation 

The second challenge facing the HPM community is validation. Validation remains a very large challenge 
for the HPMs community because statistical validation is oftentimes seen as the Holy Grail for 
determining whether a model is suitable but when models are deemed statistically valid, they generalize 
less, and are less re-usable for applications in new contexts. This places the field of modeling into the 
conundrum of making models that are statistically valid (i.e. a high correlation between predicted and 
actual data) but that lack the ability to generalize to other tasks or scenarios. When the generalizability of 
the model is limited, then its value as a cost-effective approach to predict complex human-system 
interactions is reduced. 

Validation is further challenged when modeling future technology concepts where no or little HITL data 
exists upon which to statistically validate a model (as in the NextGen aviation systems or concepts being 
designed for the Space program). It is argued that the definition of model validation must be expanded 
beyond that of statistical results validation to be more representative of a ‘model develop - model verify - 
model manipulate - model validate’ iterative process, a process that is currently underway in MIDAS’ 
FAA modeling of NextGen operations. The model develop phase of an HPM effort is one that is 
comprised primarily of model verification, where the inputs parameters such as the SEEV weights and 
workload primitives, are built from and operate as expected given the model’s context. Model verification 
is the process of determining whether a simulation model and its associated data behave as intended by 
the model developer / analyst (Sargent, 1980). The model manipulate phase is where the model’s 
conceptual parameters are manipulated to bring the overall model performance closer to expectations. The 
model validation phase involves determining the degree to which a model or simulation and its associated 
predictions are an accurate representation of the real world, from the perspective of the intended users of 
the model or simulation (Balci, 1988; Law & Kelton, 2000; Sargent, 1980). Formalized ‘develop- 
validate’ iteration cycles are an important step toward increasing the credibility of HPMs particularly as 
the complexity of human-system operations increases. 
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